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Background and Purpose 


The Local Child Outcomes Measurement System (L-COMS) is a framework developed by ECTA and DaSy that 
identifies 7 key components of a high-quality child outcomes measurement system at the local level. The following 
resources were developed for local Part C and Part B 619/Preschool programs to evaluate their systems and to 
encourage and support efforts to improve those systems. Loca/ program, as used throughout these documents, refers 
to a local/regional early intervention services program or a local educational agency or school district. 


e Local Child Outcomes Measurement System (L-COMS) —describes the framework’s 7 components, 13 quality 
indicators and their associated elements of quality 


e L-COMS Self-Assessment — an Excel-based tool that provides a structure for local programs to record the 
current status of their child outcomes measurement system on the 13 quality indicators and associated 
elements, and to set priorities for improvement 


e L-COMS Guide - contains general guidance for using the self-assessment tool 


The L-COMS was designed to apply to all local programs regardless of their 
specific child outcomes measurement approach. L-COMS Components 


Purpose 
Suggested Use of the Self-Assessment Gar dlestarend iraneniesion 


We recommend that key stakeholders in the system, including staff and Analysis 

beneficiaries of early intervention and preschool special education, complete eon 

the self-assessment. Local programs may opt to complete one or more Evaluation 

components of the self-assessment at a time or over an extended period of Cross-system Coordination 
time. Different sets of stakeholders may be appropriate participants for different 
components. If completed openly and thoughtfully by stakeholders, the self- 
assessment process will produce valuable discussions about the status of the current system and result in a 


quantitative and qualitative picture of the system’s strengths and areas for improvement. 


We encourage local programs use the self-assessment as part of an ongoing improvement planning process in which 
they assess the current status of their system, develop a plan for improvement, implement the plan, and reassess the 
system to monitor progress. When used as part of an ongoing process, reassessment allows a local program to 
monitor and document progress toward building a high-quality outcomes measurement system. Having an outcomes 
measurement system in place positions the program to be able to systematically use child outcomes data as part of 
implementing and improving high-quality supports and services for children and families. 
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Overview of the Suggested Process 

Select the component(s) for which the self-assessment will be completed. 

Identify the appropriate stakeholders for the self-assessment process for those components. 
Gather stakeholders and complete the self-assessment. 

Review the self-assessment results and determine priorities for improvement planning. 
Develop a plan for improvement. 


Implement the plan for improvement. 


NOOR OL 


Establish and maintain a continuous cycle of reassessment and improvement planning, that is, re-administer 
the self-assessment, identify areas of progress and new or ongoing areas for improvement, and revise the 
plan accordingly. 


Completing the Self-Assessment 


Each L-COMS component contains up to three Quality Indicators and each Quality Indicator has multiple Elements of 
Quality. When completing the self-assessment, stakeholders review and discuss each element, assigning it a 
consensus rating of 1 to 4, and recording brief notes (the “evidence”) describing the basis for the rating. 


Stakeholders should read each element and discuss whether the element is in place and, if so, whether it is partially or 
fully implemented. 


e A“1” on the element rating scale indicates the element is not implemented and the program is not planning to 
implement it. 

e A “2” indicates the element is not implemented but the program is planning to work on it or has started working 
on it. 

e A“3” on the scale indicates an element is partially implemented. An element can be partially implemented in a 
variety of ways. It might be that some aspects of the element are implemented and others are not. It might be 
that the element is implemented in some areas but not others. 

e Finally, a “4” on the scale indicates the element is fully implemented. By fully implemented, we mean the 
element is in place and of high quality. 


Elements of Quality Rating Scale 


1 No - element not in place and not planning to work on it at this time 
2 No - element not in place but planning to work on it or getting started 
3 Yes - element partially implemented 

4 Yes - element fully implemented 


After discussing an element, the stakeholders should determine the rating that best describes the current status of that 
element, record the rating, and document the basis for the rating. Documenting the basis for the rating decision will be 
important for improvement planning so there is a record of what the stakeholders considered in reaching their decision. 
For example, if the element was rated as partially implemented, what aspect of the element was identified as not yet in 
place or of high quality? 


The combination of ratings of the elements determines the rating for the associated Quality Indicator. Unlike the 
Element of Quality rating, the Quality Indicator rating is not selected by the stakeholder group but rather is calculated 
based on the element ratings. The rating for a Quality Indicator can range from 1 to 7, with a 1 defined as none of the 
associated Elements of Quality is planned or in place and a 7 defined as all of them are fully implemented. The self- 
assessment tool automatically calculates the Quality Indicator ratings from the element ratings. 
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Quality Indicator Rating Scale 


None of the elements is yet planned or in place. 
| 2 | Most of the elements are not yet planned or in place. 
esea Some elements are in place; a few may be fully implemented. 


[4 [Atleast hal of he elements aren place afew maybe tuly plemented, | 
5 [Atleast hal ofthe elements arin pace some af fulyimplomented. | 
[6 | Areas ha ofthe elements are fly implemented the et are paraly implemented | 


Note: In place means an element is fully or partially implemented. 


The L-COMS Self-Assessment tool creates a profile that includes a summary of the numerical ratings and a graph of 
the results. The bars in the graph are color coded to show the relative strengths and weaknesses across the Quality 
Indicators in each of the completed components. 


We encourage programs to use the self-assessment as part of an ongoing improvement planning process. The results 
of the self-assessment will help a program identify the relative strengths and weaknesses of its system, but the self- 
assessment is not a road map in that it does not tell a program where to start or what to do next. The program will 
need to determine where to focus improvement efforts based on its priorities and resources. After reviewing the overall 
results for the Quality Indicators, the stakeholders should determine the priority areas for improvement. To set 
priorities, the program can go back to the worksheet for each of the components completed. The Self-Assessment tool 
provides for priority ratings of low, medium, or high to be assigned to the Elements of Quality, the Quality Indicators, or 
both. The priorities assigned in the component worksheets are summarized on the QI Summary sheet. An 
improvement plan can be developed based on the profile and the priorities assigned by the stakeholders. 


Detailed instructions on how to use the L-COMS Self-Assessment tool are contained on the first worksheet in the file 
(“Instructions”) and on the following pages of this document. The tool can be found at ECTA Outcomes Measurement 
Frameworks or the DaSy Center L-COMS. 


Assistance with Completing the Self-Assessment 


Staff from ECTA and DaSy are available to help states plan for and support local programs in the use of the L-COMS 
Self-Assessment. This includes answering questions about how to work with the tool; assisting with planning for the 
process or facilitating the stakeholder discussion; developing processes for improvement planning; as well as providing 
other kinds of assistance the state or local programs might need. If you would like any kind of assistance or have any 
questions about the measurement framework or the self-assessment, please contact ECTA or DaSy. 


Detailed Instructions for Working in the L-COMS Self-Assessment Tool 


Overview of Instructions 


There are 12 worksheets in the L-COMS Self-Assessment tool. Seven of them are for rating the Elements of Quality 
for each of the seven components. The purpose of each worksheet is explained below along with the instructions on 
how to use it. 


Upon opening the workbook, be sure to enable the macros. There are several ways to accomplish this, depending 
upon which version of Microsoft you are using: 


e Microsoft 2013: Click on the Enable Content button in the yellow Message Bar. 
e Microsoft Excel 2007 and 2010: Go to File > Info > Enable Content > Enable All Content. 


e Microsoft 2003: Download and install the Compatibility Pack from Microsoft: https:/Awww.microsoft.com/en- 
us/download/details.aspx?id=3 
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If you cannot enable the macros through one of these methods, you may need to change a setting in your security or 
trust center settings. Additional information is provided at http://www.exceltrick.com/how_to/enable-macros-in-excel/. 


Throughout the workbook, cells that are shaded light yellow require data entry; data cannot be 


entered in cells shaded light blue; those cells are automatically calculated and cannot be edited. 


Worksheets 


The 12 Worksheets 
Instructions 
Participants 
Table of Contents 
Element Rating Sheets (7) 
Purpose 
Data Collection and Transmission 
Analysis 
Reporting 
Using Data 
Evaluation 


Cross-system Coordination 


QI Summary 
Data 


Instructions 


This sheet provides detailed instructions for working in the L-COMS Self-Assessment Tool. They are provided in this 
document and in the first sheet of the workbook for your convenience. 


Participants 


This sheet is for documenting the participants (by name and/or role) who are involved in the self-assessment. Enter 
the name and/or role of each participant, and place an “X” in the cell in the component. Enter the date on which the 
self-assessment was completed in row 12. 


Table of Contents 

This sheet lists all of the components in the L-COMS. There are links to each of the component worksheets. An 
alternative to using the links is to click on the worksheet tab at the bottom of the document. 

Element Rating Worksheets 


Element ratings: Each of these seven worksheets is used to rate the Elements of Quality (EQs) for one component. 
Within a worksheet the elements are organized by Quality Indicator (Ql). 


In the Rating column, enter the rating for each element by entering the number (1 to 4) or by selecting the value from 
the dropdown list. 


Once all element ratings for a QI are entered, the QI rating is automatically calculated and displayed in the row for the 
respective QI (in the light blue cell). 


Evidence: Immediately below each element is a text box (shaded light yellow) for documenting the evidence for the 
element. The text box will expand to the needed length, and clicking on the Show Evidence will then resize the rows 
and display all of the evidence; clicking on the Hide Evidence button will hide all of the evidence rows. Thus, the 
Element Rating worksheets may be viewed and/or printed with or without the evidence displayed. 


At the top of each sheet is a link back to the Table of Contents and buttons that make it easy to navigate to each Ql. 
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Priority: To the right of each cell for the element rating and QI rating is a cell to indicate the Priority of each element 
and QI. The Priority rating is an optional feature and may be used for planning which elements or Qls to work on. The 
Priority options are high, medium, or low, or this cell may be left blank. 


The priority designation may be entered by typing an L, M, or H (lower or upper case) or by using the dropdown list. 
The recommended process is to rate the elements of quality for all components that will be completed and then go 
back and assign the priority ratings. This allows the stakeholders to get an overall view of strengths and areas of need 
across multiple Qls and elements. 


QI Summary 


The QI Summary sheet displays a summary of the QI ratings for each of the Qls for which all the elements were rated. 
Data cannot be entered or edited on this sheet; it simply displays the summary data from the Element Rating 
worksheets. The QI ratings are displayed in column J, and the QI priority, if entered, is displayed in column K. The red, 
yellow, or green horizontal bars graphically depict the QI ratings. 


The total number of elements in the QI is presented to the right of the graph. To the right of the total is the number of 
elements in each QI with ratings of 1, 2, 3, and 4. Asummary of the number of elements with each priority is provided 
to the far right. The number of elements for each priority reflects only those elements for which a priority was selected. 


The links at the top left of the sheet, to the right of Jump To, allow easy navigation to each of the components in the QI 
Summary sheet. 


Data 


The Get Data button will generate a sheet with data for current Quality Indicator and Element ratings. If additional Qls 
are completed after getting the initial data, click on the Get Data button again to refresh the sheet with all current data. 


Note: There is a 10-15 second delay to generate the data sheet. 


Saving the Results of the Self-Assessment 


The L-COMS Self-Assessment Tool is an Excel workbook and needs to be saved just like any other workbook. We 
recommend saving the file with the date it was completed in the file name, e.g., L-COMS SA_ 102217, and then saving 
the file with a new file name if you complete more components at a later date, e.g., L-COMS SA_121517. Save the file 
often while you are working in it. 


Printing the Results of the Self-Assessment 


Each worksheet in the workbook has an existing default print area set up. The print areas may be redefined or resized 
by using the Print Area option in Excel’s Page Layout tab. 


Please cite as: 


Nelson, R., Kelley, G., Hebbeler, K., Vinh, M., Gillaspy, K., Barton, L., & Reid, J. K. (2017). L-COMS guide. Chapel Hill, 
NC: Early Childhood Technical Assistance Center. Retrieved from http://ectacenter.org/eco/assets/pdfs/L-COMS _Self- 
Assessment Guidance.pdf 


The contents of this tool and guidance were developed under grants from the U.S. Department of Education, 
#H326P120002 and #H373Z120002. However, those contents do not necessarily represent the policy of the U.S. 
Department of Education, and you should not assume endorsement by the Federal Government. 

Project Officers: Meredith Miceli, Richelle Davis, and Julia Martin Eile. 
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